Real-time computer image systems are being designed to provide realistic image reproduction for a variety of simulator systems, such as tank simulators and flight simulators. Such simulators are used as training devices which permit a combatant to obtain practice without the necessity of going out into the field and using the actual combat systems. They enable a user, such as a pilot or tank gunner, to maintain and improve his skills without the cost associated with live training. It is thus very advantageous to provide the user with video display realism which is beneficial for training and practice purposes.
In a typical real time computer image generation system, such as a flight simulator system, image generation can be broken into three separate processing stages: Controller, Geometry Processor, and Display Processor. These three processing stages or sections each work independently on data representative of or corresponding to one of three consecutive scenes to be displayed. The Controller processes data on a scene or image for a fixed time, which may be either a field time of 16.67 milliseconds or a frame time of 33.3 milliseconds, usually the former, and then passes its processed data to the Geometry Processor. This particular field and frame time corresponds to that time for a typical television format display, i.e., two interlaced fields per frame with thirty frames per second. The Geometry Processor has an equal time to do its calculations and at the end of that time its results are sent to the Display Processor. The Display Processor always processes one interlace field for a video display in one field time. For each time interval, all three processors are kept busy, so at any given time the processors are working on three separate scenes. Each section processes inputs and generates outputs during the interval, so data flows through the three stages in a sequential manner. The computational load is spread out among the specialized processor sections, so this allows new scenes to be displayed each field or frame time, even though the results are delayed three fields or frames.
For example, in a flight simulator system, pilot inputs are received by the Controller and after being processed sequentially by the Controller, the Geometry Processor and Display Processor, the output is sent as a color scene to video displays observed by the pilot in the simulator. The data seen by the pilot starts out as simple contrtol inputs applied to an environment definition and is converted to pixel video display information. In the Controller, the pilot inputs are used to calculate the new position and orientation of the aircraft, and from this, a mathematical rotation matrix is calculated that is used to rotate objects from the reference coordinate system to a display coordinate system. Any other information dealing with the aircraft, with the position of targets and with other aircraft is also calculated in the Controller. In some applications, the Controller is a general purpose computer.
The Geometry Processor reads, from a database, descriptions of objects that are potentially visible in the stored three dimensional digital representation of the scene. The objects that are read are rotated into display coordinates using the rotation matrices calculated in the Controller. Edges of these three-dimensional objects are clipped by the Geometry Processor if they extend beyond the view window boundaries. The Geometry Processor mathematically projects the three-dimensional data onto the two-dimensional display window. Object faces which extend beyond display window boundaries are then clipped in two-dimensional image space. In addition, the Geometry Processor calculates which objects are in front or behind other objects and stores this information in a priority list. Each object processed is made up of individual faces, where each face is in the form of a polygon bounded by straight edges. The priority list contains the order of all faces in the scene, with the first face in the list as the highest priority face, and the last face in the list as the lowest priority face. Whenever two faces overlap on the display, the higher priority face will be visible, and the lower priority face will be obscured. Lastly, the Geometry Processor calculates the display end points for the line segments that bound the two dimensional faces, and also calculates the coefficients of the line equations that describe the bounding edges.
The Display Processor receives the two dimensional face descriptions, along with face color, face fading information, a face priority list, cell texturing, level of detail blending, translucency, curvature shading, etc; and uses this information to output the scene to a color display. Each color display in the system is typically made up of 1024 linees of video information, with each line having 1024 individual color dots or pixels. However, other combinations of lines and pixels per line are certainly possible. The Display Processor has to resolve, for each pixel, what the color of the pixel should be, given that many faces can be present in a pixel, and that certain portions of faces may be covered by other faces. Since there can be 1024 by 1024 pixels in the display, and all these need to be calculated in 1/60th of a second, the processing load in the Display Processor is very high. To calculate video for a multi-display system (a single system has contained as many as 14 displays), the Display Processor must perform in excess of 10 bilion computations per second. Due to the tremendous processing load, a Display Processor can drive only a few displays. In some systems, a single Controller and Geometry Processor can be used to drive several Display Processors with several displays connected to each Display Processor. The Display Processor therefore represents the majority of processing in a real time computer image generation system, and for this reason most of the effort to improve the capabilities and realism of a CIG system is concentrated in the Display Processor area.
One particular problem that decreases realism is the stair-step edge that occurs in a computer generated image. This happens because the edge on the video display is caused by the discrete changes in the pixel intensity inherent in digital data. One approach to improving a video scene realism and eliminating the stair-step effect is to process the area boundaries with geometric smoothing. In geometric smoothing an area calculation is made for each edge through a pixel. However, it was found that this method would give frequent glitches in the computer generation screen if more than two edges of a face pass through a pixel.
Many display systems, particularly those employing spherical or cylindrical screens, require that a predetermined image be computed and placed on a projection raster in order that the image seen by the viewer appear correct. In general a projector and a viewer will be at different locations. If a straight line is produced on the image source, the projection raster, and projected by a standard projection lens onto the screen, it will appear curved to the observer. This is referred to as geometric distortion. If a standard F Tangent Theta lens is used to project a wide field of view seen inside a spherical screen, there will be great variation in brightness and in resolution over the region covered. The use of an F Tangent Theta lens in which the angle, Theta, between the projector boresight and a projected ray is proportional to the distance between the center of the projector raster and the point on the raster that is the source of the ray, provides approximately uniform brightness and resolution. With such a lens, even if the projector and viewer were co-located, a scene correct on the projector raster would appear distorted to the viewer, that is, optically distorted.
It is, therefore, an object of this invention to provide a comprehensive distortion correction technique which results in a predistorted scene on a projector raster which when projected, looks correct to the viewer and simultaneously corrects the geometric distortion and optical distortion.
It is an object of the present invention to provide a method for Display Processor calculations that reduces the processing in a real time computer image generation system while decreasing the computational time of the Display Processor.
It is a further object to provide a method in which an unlimited number of edges can be processed through a pixel, to provide in real time a noise-free video display.